Extensive testing is vital to the successful development of software applications. If a software application is not properly tested, the end user may experience software bugs that hinder efficiency, cause potentially fatal data loss, or generally render the software application frustrating to use, among other problems. As a result, software developers allocate substantial human and monetary resources to software testing in an effort to ensure that the end user experience mirrors that intended by the software developer.
For some software applications, it is common for thousands or even millions of tests to be conducted on a periodic basis (e.g., weekly) to verify that the software application functions properly at various stages of development. Due to the number of tests that must be performed, these tests are often automated and managed by an automated test system. Reporting failures that result from automated tests is one function of an automated test system. In some implementations, when a test fails, an automated test system stores a result that then needs to be further investigated by a human tester. As software applications become increasingly complex, more tests are typically needed to validate the application, and those tests are run more often. This increases the number of automation failures that may occur and, as a result, the volume of automation being run generates more results than human tester resources can effectively investigate. This results in a number of potential issues that are identified, but nobody is available to perform any further investigation. These potential issues may be inadvertently included in a version of the software application that reaches end users.
As the volume of automated tests increases, it becomes increasingly more difficult for testers to keep up with the volume of data produced. In response to this, “failure tagging” was created. Failure tagging allows testers to specify a pattern occurring in specific tests which represent identical problems. If a pattern reoccurs, a tester does not need to re-investigate that failure. Failure Tagging depends on testers to generate accurate information about the failure for it to function well, so when testers create a poor “tag”, the system may flag unrelated failures as a recurrence of a tagged failure.
Managing a large set of failure tags is time intensive. That is, as testers have more results to investigate, they have less time available to manage failure tags, which, in turn, increases the chances of a poor tag being created because users are rushing through the investigation process. Failure tags are also not resilient to changes in test logging and do not properly function for automation executed for multiple languages. Even with failure tagging, the failure investigation rate can be under 50%.
Software applications have been developed that identify a crash in a software application and uniquely identify that crash. Information related to the crash is then placed into a “bucket”. If the same crash occurs again, the information from that crash is added to the same bucket. This functionality is based off of what happens in an end-user application, not an application in development and undergoing testing. Additionally, this functionality identifies buckets based off of information, such as the version of the software application. This does not change frequently for software applications that have been released to the end user. However, for software applications that are under development, the version can change on a daily or even hourly basis, thus generating many buckets for a single problem.
It is with respect to these and other considerations that the disclosure made herein is presented.